Incremental multi-linear discriminant analysis using canonical correlations for action recognition

Canonical correlations analysis (CCA) is often used for feature extraction and dimensionality reduction. However, the image vectorization in CCA breaks the spatial structure of the original image, and the excessive dimension of vector often brings the curse of dimensionality problem. In this paper, we propose a novel feature extraction method based on CCA in multi-linear discriminant subspace by encoding an action sample as a high-order tensor. An optimization approach is presented to iteratively learn the discriminant subspace by unfolding the tensor along different tensor modes. It retains most of the underlying data structure including the spatio-temporal information, and alleviates the curse of dimensionality problem. At the same time, an incremental scheme is developed for multi-linear subspace online learning, which can improve the discriminative capability efficiently and effectively. The nearest neighbor classifier (NNC) is exploited for action classification. Experiments on Weizmann database showed that the proposed method outperforms the state-of-the-art methods in terms of accuracy. The proposed method has low time complexity and is robust against partial occlusion.